大型变压器模型在许多任务中产生令人印象深刻的结果，但培训昂贵，甚至微调，如此慢，在解码中，他们的使用和研究变得无法触及。我们通过利用稀疏性来解决这个问题。我们研究变压器中的所有层的稀疏变体，并提出缩放变压器，一个缩放变压器模型，使用稀疏层的型号有效地缩放，并在我们扩展模型大小时比标准变压器更快地执行不匹配的解码。令人惊讶的是，稀疏层足以获得与具有相同数量的参数的标准变压器相同的困惑。我们还与现有的稀疏性融合，即使存储器有限，也能够对长期序列进行快速推断。这导致在长期摘要上对最先进的表现竞争。

We introduce Performers, Transformer architectures which can estimate regular (softmax) full-rank-attention Transformers with provable accuracy, but using only linear (as opposed to quadratic) space and time complexity, without relying on any priors such as sparsity or low-rankness. To approximate softmax attentionkernels, Performers use a novel Fast Attention Via positive Orthogonal Random features approach (FAVOR+), which may be of independent interest for scalable kernel methods. FAVOR+ can also be used to efficiently model kernelizable attention mechanisms beyond softmax. This representational power is crucial to accurately compare softmax with other kernels for the first time on large-scale tasks, beyond the reach of regular Transformers, and investigate optimal attention-kernels. Performers are linear architectures fully compatible with regular Transformers and with strong theoretical guarantees: unbiased or nearly-unbiased estimation of the attention matrix, uniform convergence and low estimation variance. We tested Performers on a rich set of tasks stretching from pixel-prediction through text models to protein sequence modeling. We demonstrate competitive results with other examined efficient sparse and dense attention methods, showcasing effectiveness of the novel attention-learning paradigm leveraged by Performers.

This paper presents a novel federated reinforcement learning (Fed-RL) methodology to enhance the cyber resiliency of networked microgrids. We formulate a resilient reinforcement learning (RL) training setup which (a) generates episodic trajectories injecting adversarial actions at primary control reference signals of the grid forming (GFM) inverters and (b) trains the RL agents (or controllers) to alleviate the impact of the injected adversaries. To circumvent data-sharing issues and concerns for proprietary privacy in multi-party-owned networked grids, we bring in the aspects of federated machine learning and propose a novel Fed-RL algorithm to train the RL agents. To this end, the conventional horizontal Fed-RL approaches using decoupled independent environments fail to capture the coupled dynamics in a networked microgrid, which leads us to propose a multi-agent vertically federated variation of actor-critic algorithms, namely federated soft actor-critic (FedSAC) algorithm. We created a customized simulation setup encapsulating microgrid dynamics in the GridLAB-D/HELICS co-simulation platform compatible with the OpenAI Gym interface for training RL agents. Finally, the proposed methodology is validated with numerical examples of modified IEEE 123-bus benchmark test systems consisting of three coupled microgrids.

在农业环境中的现代除草剂应用通常依赖于将除草剂分配给作物和杂草相似的或便携式喷雾器的大型喷雾器，这些喷雾器需要劳动密集型手动操作。前一种方法导致过度使用除草剂并减少作物产量，而后者在大规模操作中经常站立。本文介绍了能够基于计算机视觉的导航，杂草检测，完整的现场覆盖以及\ $ 400下的计算机视觉的行作物的杂草管理的第一个完全自主机器人。目标应用程序是在裁剪领域中的自主行行杂草控制，例如，亚麻和油菜，在农作物之间的间距像一只脚一样小。所提出的机器人足够小，可以在植物生长的所有阶段之间通过植物生长的阶段，同时检测杂草和喷洒除草剂。充电系统包括新设计的机器人硬件，斜坡，机器人充电臂和移动充电站。采用集成视觉算法，有效地帮助充电器对齐。结合，它们使机器人能够在现场中连续工作而不获得电力。此外，将与预处理技术相结合的基于颜色的轮廓算法用于依赖于从车载单手套摄像机的输入上的鲁棒导航。将这种紧凑的机器人纳入农场可以帮助自动化杂草控制，即使在增长的后期阶段，并通过精确定位杂草减少除草剂。机器人平台在北达科他州的亚麻籽领域进行了现场测试。

全球DataSphere快速增加，预计将达到20251年的175个Zettabytes。但是，大多数内容都是非结构化的，并且无法通过机器可以理解。将此数据构建到知识图中，使得智能应用程序具有诸如深度问题的智能应用，推荐系统，语义搜索等。知识图是一种新兴技术，允许使用内容与上下文一起逻辑推理和揭示新的洞察。因此，它提供了必要的语法和推理语义，使得能够解决复杂的医疗保健，安全，金融机构，经济学和业务问题。作为一项结果，企业正在努力建设和维护知识图表，以支持各种下游应用。手动方法太贵了。自动化方案可以降低建设知识图的成本，高达15-250次。本文批评了最先进的自动化技术，以自主地生成近乎人类的近乎人类的质量。此外，它突出了需要解决的不同研究问题，以提供高质量的知识图表

Spear Phishing is a harmful cyber-attack facing business and individuals worldwide. Considerable research has been conducted recently into the use of Machine Learning (ML) techniques to detect spear-phishing emails. ML-based solutions may suffer from zero-day attacks; unseen attacks unaccounted for in the training data. As new attacks emerge, classifiers trained on older data are unable to detect these new varieties of attacks resulting in increasingly inaccurate predictions. Spear Phishing detection also faces scalability challenges due to the growth of the required features which is proportional to the number of the senders within a receiver mailbox. This differs from traditional phishing attacks which typically perform only a binary classification between phishing and benign emails. Therefore, we devise a possible solution to these problems, named RAIDER: Reinforcement AIded Spear Phishing DEtectoR. A reinforcement-learning based feature evaluation system that can automatically find the optimum features for detecting different types of attacks. By leveraging a reward and penalty system, RAIDER allows for autonomous features selection. RAIDER also keeps the number of features to a minimum by selecting only the significant features to represent phishing emails and detect spear-phishing attacks. After extensive evaluation of RAIDER over 11,000 emails and across 3 attack scenarios, our results suggest that using reinforcement learning to automatically identify the significant features could reduce the dimensions of the required features by 55% in comparison to existing ML-based systems. It also improves the accuracy of detecting spoofing attacks by 4% from 90% to 94%. In addition, RAIDER demonstrates reasonable detection accuracy even against a sophisticated attack named Known Sender in which spear-phishing emails greatly resemble those of the impersonated sender.